Holomorphic extensions of representations : ( I ) automorphic functions

Let G be a connected, real, semisimple Lie group contained in its complexification GC, and let K be a maximal compact subgroup of G. We construct a KC-G double coset domain in GC, and we show that the action of G on the K-finite vectors of any irreducible unitary representation of G has a holomorphic extension to this domain. For the resultant holomorphic extension of K-finite matrix coefficients we obtain estimates of the singularities at the boundary, as well as majorant/minorant estimates along the boundary. We obtain L∞ bounds on holomorphically extended automorphic functions on G/K in terms of Sobolev norms, and we use these to estimate the Fourier coefficients of combinations of automorphic functions in a number of cases, e.g. of triple products of Maaß forms. Introduction Complex analysis played an important role in the classical development of the theory of Fourier series. However, even for Sl(2,R) contained in Sl(2,C), complex analysis on Sl(2,C) has had little impact on the harmonic analysis of Sl(2,R). As the K-finite matrix coefficients of an irreducible unitary representation of Sl(2,R) can be identified with classical special functions, such as hypergeometric functions, one knows they have holomorphic extensions to some domain. So for any infinite dimensional irreducible unitary representation of Sl(2,R), one can expect at most some proper subdomain of Sl(2,C) to occur. It is less clear that there is a universal domain in Sl(2,C) to which the action of G on K-finite vectors of every irreducible unitary representation has holomorphic extension. One goal of this paper is to construct such a domain for a real, connected, semisimple Lie group G contained in its complexification GC. It is important to have a maximal domain, and towards this goal we show that this one is maximal in some directions. ∗The first named author was supported in part by NSF grant DMS-0097314. The second named author was supported in part by NSF grant DMS-0070742. 642 BERNHARD KRÖTZ AND ROBERT J. STANTON Although defined in terms of subgroups of GC, the domain is natural also from the geometric viewpoint. This theme is developed more fully in [KrStII] where we show that the quotient of the domain by KC is bi-holomorphic to a maximal Grauert tube of G/K with the adapted complex structure, and where we show that it also contains a domain bi-holomorphic but not isometric with a related bounded symmetric domain. Some implications of this for the harmonic analysis of G/K are also developed there. However, the main goal of this paper is to use the holomorphic extension of K-finite vectors and their matrix coefficients to obtain estimates involving automorphic functions. To our knowledge, Sarnak was the first to use this idea in the paper [Sa94]. For example, with it he obtained estimates on the Fourier coefficients of polynomials of Maaß forms for G = SO(3, 1). Sarnak also conjectured the size of the exponential decay rate for similar coefficients for Sl(2,R). Motivated by Sarnak’s work, Bernstein-Reznikov, in [BeRe99], verified this conjecture, and in the process introduced a new technique involving G-invariant Sobolev norms. As an application of the holomorphic extension of representations and with a more representation-theoretic treatment of invariant Sobolev norms, we shall verify a uniform version of the conjecture for all real rank-one groups. As the representation-theoretic techniques are general, we are able also to obtain estimates for the decay rate of Fourier coefficients of Rankin-Selberg products of Maaß forms for G = Sl(n,R), and to give a conceptually simple proof of results of Good, [Go81a,b], on the growth rate of Fourier coefficients of Rankin-Selberg products for co-finite volume lattices in Sl(2,R). It is a pleasure to acknowledge Nolan Wallach’s influence on our work by his idea of viewing automorphic functions as generalized matrix coefficients, and to thank Steve Rallis for bringing the Bernstein-Reznikov work to our attention, as well as for encouraging us to pursue this project. To the referee goes our gratitude for a careful reading of our manuscript that resulted in the correction of some oversights, as well as a notable improvement of our estimates on automorphic functions for Sl(3,R). 1. The double coset domain To begin we recall some standard structure theory in order to be able to define the domain that will be important for the rest of the paper. Any standard reference for structure theory, such as [Hel78], is adequate. Let g be a real, semisimple Lie algebra with a Cartan involution θ. Denote by g = k ⊕ p the associated Cartan decomposition. Take a ⊆ p a maximal abelian subspace and let Σ = Σ(g, a) ⊆ a∗ be the corresponding root system. Related to this root system is the root space decomposition according to the simultaneous eigenvalues of ad(H),H ∈ a : HOLOMORPHIC EXTENSIONS OF REPRESENTATIONS I 643 g = a ⊕ m ⊕ ⊕ α∈Σ g; here m = zk(a) and g α = {X ∈ g: (∀H ∈ a) [H,X] = α(H)X}. For the choice of a positive system Σ+ ⊆ Σ one obtains the nilpotent Lie algebra n = ⊕ α∈Σ+ g α. Then one has the Iwasawa decomposition on the Lie algebra level g = k ⊕ a ⊕ n. Let GC be a simply connected Lie group with Lie algebra gC, where for a real Lie algebra l, by lC we mean its complexification. We denote by G,A,AC,K,KC, N and NC the analytic subgroups of GC corresponding to g, a, aC, k, kC, n and nC. If u = k ⊕ ip then it is a subalgebra of gC and the corresponding analytic subgroup U = exp(u) is a maximal compact, and in this case, simply connected, subgroup of GC. For these choices one has for G the Iwasawa decomposition, that is, the multiplication map K ×A×N → G, (k, a, n) 7→ kan is an analytic diffeomorphism. In particular, every element g ∈ G can be written uniquely as g = κ(g)a(g)n(g) with each of the maps κ(g) ∈ K, a(g) ∈ A, n(g) ∈ N depending analytically on g ∈ G. We shall be concerned with finding a suitable domain in GC on which this decomposition extends holomorphically. Of course, various domains having this property have been obtained by several individuals. What distinguishes the one here is its KC-G double coset feature as well as a type of maximality. First we note the following: Lemma 1.1. The multiplication mapping Φ:KC ×AC ×NC → GC, (k, a, n) 7→ kan has everywhere surjective differential. Proof. Obviously one has gC = kC ⊕ aC ⊕ nC and aC ⊕ nC is a subalgebra of gC. Then following Harish-Chandra, since Φ is left KC and right NC-equivariant it suffices to check that dΦ(1, a,1) is surjective for all a ∈ AC. Let ρa(g) = ga be the right translation in GC by the element a. Then for X ∈ kC, Y ∈ aC and Z ∈ nC one has dΦ(1, a,1)(X,Y,Z) = dρa(1)(X + Y +Ad(a)Z), from which the surjectivity follows. To describe the domain we extend a to a θ-stable Cartan subalgebra h of g so that h = a ⊕ t with t ⊆ m. Let ∆ = ∆(gC, hC) be the corresponding root system of g. Then it is known that ∆ |a\{0} = Σ. 644 BERNHARD KRÖTZ AND ROBERT J. STANTON Let Π = {α1, . . . , αn} be the set of simple restricted roots corresponding to the positive roots Σ+. We define elements ω1, . . . , ωn of a ∗ as follows, using the restriction of the Cartan-Killing form to a: (∀1 ≤ i, j ≤ n)    〈ωj, αi〉 = 0 if i 6= j 2〈ωi,αi〉 〈αi,αi〉 = 1 if αi ∈ ∆ 〈ωi,αi〉 〈αi,αi〉 = 1 if αi 6∈ ∆ and 2αi 6∈ Σ 〈ωi,αi〉 〈αi,αi〉 = 2 if αi 6∈ ∆ and 2αi ∈ Σ. Using standard results in structure theory relating ∆ and Σ one can show that ω1, . . . , ωn are algebraically integral for ∆ = ∆(gC, hC). The last piece of structure theory we shall recall is the little Weyl group. We denote by Wa = NK(a)/ZK(a) the Weyl group of Σ(a, g). We are ready to define a first approximation to the double coset domain. We set aC = {X ∈ aC: (∀1 ≤ k ≤ n)(∀w ∈ Wa) | Imωk(w.X)| < π 4 } and aC = 2a 1 C. On the group side we let A0 C = exp(a0 C ) and A1 C = exp(a1 C ). Clearly Wa leaves each of a0 C , a1 C , A0 C and A1 C invariant. If α ∈ a∗ C is analytically integral for AC, then we set a α = eα(log a) for all a ∈ AC. Since GC is simply connected, the elements ωj are analytically integral for AC and so we have a ωk well defined. Next we introduce the domains A0,≤ C = {a ∈ AC: (∀1 ≤ k ≤ n)Re(ak) > 0}, and A1,≤ C = (A0,≤ C ) 1 2 = {a ∈ AC: (∀1 ≤ k ≤ n)| arg(ak)| < π 4 }. Note that A0 C ⊆ A0,≤ C and A1 C ⊆ A1,≤ C . Lemma 1.2. (i) For Ω ⊆ AC open, KCΩNC is open in GC. In particular, the sets KCACNC, KCA 1 C NC, KCA 1,≤ C NC, KCA 0 C NC and KCA 0,≤ C NC are open in GC. (ii) KCACNC is dense in GC. Proof. This is an immediate consequence of Lemma 1.1 as Φ is a morphism of affine algebraic varieties with everywhere submersive differential. Proposition 1.3. Let GC be a simply connected, semisimple, complex Lie group. Then the multiplication mapping Φ:KC ×A0,≤ C ×NC → GC, (k, a, n) 7→ kan is an analytic diffeomorphism onto its open image KCA 0,≤ C NC. HOLOMORPHIC EXTENSIONS OF REPRESENTATIONS I 645 Proof. In view of the preceding lemmas, it suffices to show that Φ is injective. Suppose that kan = k′a′n′ for some k, k′ ∈ KC, a, a′ ∈ A0,≤ C and n, n′ ∈ NC. Denote by Θ the holomorphic extension of the Cartan involution of G to GC. Then we get that Θ(kan)−1kan = Θ(k′a′n′)−1k′a′n′ or equivalently Θ(n−1)a2n = Θ((n′)−1)(a′)2n′. Now the subgroup NC = Θ(NC) corresponds to the analytic subgroup with Lie algebra nC = ⊕ α∈−Σ+ g α C . As a consequence of the injectivity of the map NC ×AC ×NC → NCACNC, (n, a, n) 7→ nan we conclude that n = n′ and a2 = (a′)2. We may assume that a, a′ ∈ exp(ia). To complete the proof of the proposition it remains to show that a2 = (a′)2 for a, a′ ∈ A0,≤ C implies that a = a′. Let X1, . . . ,Xn in aC be the dual basis to ω1, . . . , ωn. We can write a = exp( ∑n j=1φjXj) and a ′ = exp( ∑n j=1 φ ′ jXj) for complex numbers φj , φ ′ j satisfying | Imφj | < π2 , | Imφj | < π2 . Then a2 = (a′)2 implies that ej = aj = (a′)2ωj = e ′ j and hence φj = φ ′ j for all 1 ≤ j ≤ n, concluding the proof of the proposition. Thus every element z ∈ KCA C NC can be uniquely written as z = κ(z)a(z)n(z) with κ(z) ∈ KC, a(z) ∈ A0,≤ C and n(z) ∈ NC all depending holomorphically on z. Next we define domains using the restricted roots. We set b = {X ∈ a: (∀α ∈ Σ) |α(X)| < π}. and b = 1 2 b. Clearly both b0 and b1 areWa-invariant. We set bjC = a+ibj andB j C = exp(b C ) for j = 0, 1. Let a0 = i(a0 C ∩ ia). Then, from the classification of restricted root systems and standard facts about the associated fundamental weights, one can verify that a0 ⊆ b0. For a comparison of these domains we provide below the illustrations for two rank 2 algebras. Lemma 1.4. Let ω ⊆ ib1 be a nonempty, open, Wa-invariant, convex set. Then the set KC exp(ω)G is open in GC. 646 BERNHARD KRÖTZ AND ROBERT J. STANTON Figure 1 Figure 2 π 2 Hα 2 ω2